Games for the Optimal Deployment of Security Forces

In this thesis, we develop mathematical models for the optimal deployment of security forces addressing two main challenges: adaptive behavior of the adversary and uncertainty in the model. We address several security applications and model them as agent-intruder games. The agent represents the security forces which can be the coast guard, airport control, or military assets, while the intruder represents the agent's adversary such as illegal fishermen, terrorists or enemy submarines. To determine the optimal agent's deployment strategy, we assume that we deal with an intelligent intruder. This means that the intruder is able to deduce the strategy of the agent. To take this into account, for example by using randomized strategies, we use game theoretical models which are developed to model situations in which two or more players interact. Additionally, uncertainty may arise at several aspects. For example, there might be uncertainty in sensor observations, risk levels of certain areas, or travel times. We address this uncertainty by combining game theoretical models with stochastic modeling, such as queueing theory, Bayesian beliefs, and stochastic game theory. This thesis consists of three parts. In the first part, we introduce two game theoretical models on a network of queues. First, we develop an interdiction game on a network of queues where the intruder enters the network as a regular customer and aims to route to a target node. The agent is modeled as a negative customer which can inspect the queues and remove intruders. By modeling this as a queueing network, stochastic arrivals and travel times can be taken into account. The second model considers a non-cooperative game on a queueing network where multiple players decide on a route that minimizes their sojourn time. We discuss existence of pure Nash equilibria for games with continuous and discrete strategy space and describe how such equilibria can be found. The second part of this thesis considers dynamic games in which information that becomes available during the game plays a role. First, we consider partially observable agent-intruder games (POAIGs). In these types of games, both the agent and the intruder do not have full information about the state space. However, they do partially observe the state space, for example by using sensors. We prove the existence of approximate Nash equilibria for POAIGs with an infinite time horizon and provide methods to find (approximate) solutions for both POAIGs with a finite time horizon and POAIGs with an infinite time horizon. Second, we consider anti-submarine warfare operations with time dependent strategies where parts of the agent's strategy becomes available to the intruder during the game. The intruder represents an enemy submarine which aims to attack a high value unit. The agent is trying to prevent this by the deployment of both frigates and helicopters. In the last part of this thesis we discuss games with restrictions on the agent's strategy. We consider a special case of security games dealing with the protection of large areas for a given planning period. An intruder decides on which cell to attack and an agent selects a patrol route visiting multiple cells from a finite set of patrol routes, such that some given operational conditions on the agent's mobility are met. First, this problem is modeled as a two-player zero-sum game with probabilistic constraints such that the operational conditions are met with high probability. Second, we develop a dynamic variant of this game by using stochastic games. This ensures that strategies are constructed that consider both past actions and expected future risk levels. In the last chapter, we consider Stackelberg security games with a large number of pure strategies. In order to construct operationalizable strategies we limit the number of pure strategies that is allowed in the optimal mixed strategy of the agent. We investigate the cost of these restrictions by introducing the price of usability and develop algorithmic approaches to calculate such strategies efficiently

The price of usability: Designing operationalizable strategies for security games

We consider the problem of allocating scarce security resources among heterogeneous targets to thwart a possible attack. It is well known that deterministic solutions to this problem being highly predictable are severely suboptimal. To mitigate this predictability, the game-theoretic security game model was proposed which randomizes over pure (deterministic) strategies, causing confusion in the adversary. Unfortunately, such mixed strategies typically randomize over a large number of strategies, requiring security personnel to be familiar with numerous protocols, making them hard to operationalize. Motivated by these practical considerations, we propose an easy to use approach for computing strategies that are easy to operationalize and that bridge the gap between the static solution and the optimal mixed strategy. These strategies only randomize over an optimally chosen subset of pure strategies whose cardinality is selected by the defender, enabling them to conveniently tune the trade-off between ease of operationalization and efficiency using a single design parameter. We show that the problem of computing such operationalizable strategies is NP-hard, formulate it as a mixed-integer optimization problem, provide an algorithm for computing ϵ-optimal equilibria, and an efficient heuristic. We evaluate the performance of our approach on the problem of screening for threats at airport checkpoints and show that the Price of Us-ability, i.e., the loss in optimality to obtain a strategy that is easier to operationalize, is typically not high

Static and dynamic appointment scheduling to improve patient access time

Appointment schedules for outpatient clinics have great influence on efficiency and timely access to health care services. The number of new patients per week fluctuates, and capacity at the clinic varies because physicians have other obligations. However, most outpatient clinics use static appointment schedules, which reserve capacity for each patient type. In this paper, we aim to optimise appointment scheduling with respect to access time, taking fluctuating patient arrivals and unavailabilities of physicians into account. To this end, we formulate a stochastic mixed integer programming problem, and approximate its solution invoking two different approaches: (1) a mixed integer programming approach that results in a static appointment schedule, and (2) Markov decision theory, which results in a dynamic scheduling strategy. We apply the methodologies to a case study of the surgical outpatient clinic of the Jeroen Bosch Hospital. We evaluate the effectiveness and limitations of both approaches by discrete event simulation; it appears that allocating only 2% of the capacity flexibly already increases the performance of the clinic significantly

Offload zone patient selection criteria to reduce ambulance offload delay

Emergency department overcrowding is a widespread problem and often leads to ambulance offload delay. If no bed is available when a patient arrives, the patient has to wait with the ambulance crew. A recent Canadian innovation is the offload zone—an area where multiple patients can wait with a single paramedic–nurse team allowing, the ambulance crew to return to service immediately. Although a reduction in offload delay was anticipated, it was observed that the offload zone is often at capacity. In this study we investigate why this is the case and use a continuous time Markov chain to evaluate how interventions can prevent congestion in the offload zone. Specifically we demonstrate conditions where the offload zone worsens offload delay and conditions where the offload zone can essentially eliminate offload delay

Security games with restricted strategies: an approximate dynamic programming approach

In this chapter we consider a security game between an agent and an intruder to find optimal strategies for patrolling against illegal fishery. When patrolling large areas that consist of multiple cells, several aspects have to be taken into account. First, the current risk of the cells has to be considered such that cells with high risk are visited more often. Moreover, it is important to be unpredictable in order to increase the patrolling effectiveness countering illegal fishery. Finally, patrolling strategies have to be chosen in such a manner that they satisfy given operational requirements. For example, the agent might be required to patrol some cells more often than others imposing extra restrictions on the agent strategies. In this chapter, we develop a dynamic variant of the security game with restrictions on the agents strategy so that all these requirements are taken into account. We model this game as a stochastic game with a final penalty to ensure that the operational requirements are met. In this way, strategies are formed that both consider past actions and expected future risk levels. Due to the curse of dimensionality, these stochastic games cannot be solved for large scale instances. We develop an approximate dynamic programming algorithm to find approximate solutions

An interdiction game on a queueing network with multiple intruders

Security forces are deployed to protect networks that are threatened by multiple intruders. To select the best deployment strategy, we analyze an interdiction game that considers multiple simultaneous threats. Intruders route through the network as regular customers, while interdictors arrive at specific nodes as negative customers. When an interdictor arrives at a node where an intruder is present, the intruder is removed from the network. Intruders and interdictors compete over the value of this network, which is the throughput of unintercepted intruders. Intruders attempt to maximize this throughput by selecting a fixed route through the network, while the interdictors aim to minimize the throughput selecting their arrival rate at each node. We analyze this game and characterize optimal strategies. For special cases, we obtain explicit formulas to evaluate the optimal strategies and use these to compute optimal strategies for general networks. We also consider the network with probabilistic routing of intruders and show that for this case, the value and optimal strategies of the interdictor of the resulting game remain the same